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1.  Abstract 

In  sampling-based  reliability-based  design  optimization  of  practical  complex  engineering  applications,  the  Monte 
Carlo  simulation  (MCS)  for  stochastic  sensitivity  analysis  and  probability  of  failure  calculation  is  based  on  the 
prediction  from  the  surrogate  model  for  the  performance  functions.  When  the  number  of  samples  used  to  construct 
the  surrogate  model  is  small,  the  prediction  from  the  surrogate  model  becomes  inaccurate  and  thus  MCS  becomes 
inaccurate  as  well.  Therefore,  to  count  in  the  prediction  error  from  the  surrogate  model  and  assure  the  obtained 
optimum  design  from  sampling-based  RBDO  does  satisfy  the  probabilistic  constraints,  a  conservative  surrogate 
model  is  needed.  In  this  paper,  a  conservative  surrogate  model  is  constructed  using  the  weighted  Kriging  variance 
where  the  weight  is  determined  by  the  relative  change  in  the  corrected  Akaike  information  criterion  (AICc)  of  the 
dynamic  Kriging  model.  The  proposed  conservative  surrogate  model  performs  better  than  the  traditional  Kriging 
prediction  interval  approach  because  it  does  not  generate  unnecessary  local  optimum;  and  it  performs  better  than 
the  constant  safety  margin  approach  because  it  adaptively  counts  in  the  uncertainty  of  the  surrogate  model  in  the 
place  that  the  samples  are  sparse.  Numerical  examples  show  that  using  the  proposed  conservative  surrogate  model 
for  sampling-based  RBDO  is  necessary  to  assure  the  optimum  design  satisfies  the  probabilistic  constraints  when 
the  number  of  samples  is  limited. 

2.  Keywords:  conservative  surrogate  model,  dynamic  Kriging  method,  weighted  Kriging  variance,  corrected 
Akaike  information  criterion  (AICc). 

3.  Introduction 

In  sampling-based  RBDO,  Monte  Carlo  simulation  (MCS)  is  used  to  carry  out  both  the  probability  of  failure  and 
the  stochastic  sensitivity  analysis  [1,2].  To  carry  out  MCS  efficiently  in  complex  engineering  applications,  a 
surrogate  model  is  used  to  predict  the  performance  function  at  the  MCS  samples.  Zhao  et  al.  [3]  developed  a 
dynamic  Kriging  method  to  accurately  construct  the  prediction  of  the  performance  function  for  the  MCS  by  using 
the  genetic  algorithm  for  the  basis  function  selection  and  the  generalized  pattern  search  for  the  correlation 
parameter  estimation.  When  the  surrogate  model  is  not  accurate  enough,  new  samples  are  sequentially  inserted  to 
improve  the  prediction  accuracy  [3],  However,  inserting  a  large  number  of  new  samples  sequentially  may  not  be 
applicable  when  the  samples  are  from  the  physical  experiments  or  when  the  computational  resource  for  simulation 
is  limited.  Therefore,  the  improvement  of  the  surrogate  model  cannot  be  from  the  increase  of  the  sample  numbers. 
When  applying  the  surrogate  model  for  reliability-based  design  optimization  when  the  number  of  samples  is  small, 
to  assure  the  obtained  optimum  design  can  satisfy  the  probabilistic  constraints,  a  conservative  surrogate  model  is 
needed. 

Picheny  [4]  used  both  safety  margin  and  safety  factor  approaches  to  construct  the  conservative  surrogate  model 
and  concluded  that  when  the  Kriging  method  is  used,  both  methods  provide  similar  performance  in  terms  of  the 
conservativeness  and  the  accuracy.  Hertog  et  al.  [5]  and  Luna  and  Young  [6]  used  the  bootstrapping  method  to 
estimate  the  Kriging  prediction  interval  to  construct  the  conservative  surrogate  model.  The  bootstrapping  variance 
is  larger  than  the  traditional  Kriging  prediction  variance  by  considering  the  uncertainty  from  the  correlation 
parameter  in  the  Kriging  method.  However,  the  bootstrapping  procedure  is  time-consuming  and  not  applicable  for 
high-dimensional  problems.  Viana  et  al.  [7]  used  cross-validation  to  estimate  the  safety  margin  for  the 


conservative  surrogate  model.  While  the  cross-validation  error  is  widely  used  to  estimate  the  prediction  error,  this 
constant  safety  margin  approach  does  not  distinguish  the  prediction  error  at  different  sample  locations  and  will 
yield  an  over-conservative  surrogate  model  where  the  samples  are  aggregated  around;  and  an  under-conservative 
surrogate  model  where  the  samples  are  sparse. 

In  this  paper,  a  weighted  Kriging  variance  is  considered  to  construct  a  more  accurate  conservative  surrogate  model 
based  the  sample  locations.  To  evaluate  the  importance  from  each  sample  and  determine  the  weight  for  it,  an 
accuracy  measure  of  the  surrogate  model  is  needed  first.  Under  the  Kriging  framework,  the  Akaike  infonnation 
criterion  (AIC)  [8]  is  a  reliable  indicator  to  assess  the  accuracy  of  the  surrogate  model.  Hurvich  and  Tsai  [9] 
proposed  a  corrected  AIC  (AICc)  to  correct  the  bias  in  AIC  when  the  number  of  samples  is  small.  Burnham  and 
Anderson  [  1 0]  recommended  using  AICc  rather  than  AIC  when  the  sample  size  is  small  and  showed  that  the  AICc 
converges  to  AIC  as  the  number  of  samples  increases.  Martin  and  Simpson  [11]  compared  the  performance  of  the 
corrected  AIC  assessment  with  other  methods  and  concluded  that  AICc  provides  the  best  accuracy  assessment  for 
surrogate  model  accuracy.  In  this  paper,  the  AICc  is  used  to  assess  the  accuracy  changes  in  the  surrogate  model 
during  the  cross-validation  process,  and  an  importance  function  value  is  assigned  to  each  sample  according  to  the 
relative  changes  in  AICc  of  the  surrogate  model  and  the  weight  is  also  assigned  to  each  sample  according  to  the 
importance  function  values.  Then  a  weighted  Kriging  variance  is  calculated  based  on  the  weight  values  for  each 
sample.  By  applying  this  weighted  Kriging  variance,  one  can  construct  the  conservative  surrogate  model  for 
dynamic  Kriging  and  use  it  for  sampling-based  RBDO  to  assure  the  obtained  optimum  design  can  satisfy  the 
probabilistic  constraint. 

The  remainder  of  this  paper  is  organized  in  four  sections.  First,  the  backgrounds  of  the  dynamic  Kriging  method, 
sampling-based  RBDO,  and  the  AICc  are  briefly  summarized.  Then,  the  weighted  Kriging  variance  using  the 
weight  from  the  relative  change  in  AICc  is  introduced  to  construct  the  conservative  surrogate  model.  Third,  the 
conservative  surrogate  model  is  applied  to  sampling-based  RBDO  and  the  optimum  design  is  verified  for  the 
probabilistic  constraint  using  numerical  examples  and  compared  with  the  results  using  the  constant  safety  margin 
approach. 

4.  Background  of  the  Dynamic  Kriging  Method,  Sampling-based  RBDO,  and  AICc 

4.1  Dynamic  Kriging  Method 

First  consider  a  universal  Kriging  method.  The  outcomes  are  considered  as  a  realization  of  a  stochastic  process, 
and  the  predicted  values  are  derived  by  applying  the  stochastic  process  theory.  Consider  n  sample  points 
X  =  [x,,x2,...,x„]T withx,  eRnd,  and  n  responsesY  =  [y(x1),y(x2),...,y(xn)]T  withy(x;)eR.  In  the  Kriging 
method,  the  responses  at  samples  are  considered  as  a  summation  of  two  parts  as 

Y  =  Fp  +  e  (1) 

The  first  part  of  the  right-hand  side  of  Eq.  (1),  Fp,  is  considered  as  the  mean  structure  of  the  response,  where 
F=[f(xi),/  =  1,...,»]  is  a  (nxK)  design  matrix,  and  f(x)  =  [ fk(\),k  =  1,...,  K]  represents  the  user-defined  basis 
functions,  which  are  usually  in  a  simple  polynomial  form,  such  as  1,  jc,  x2 .  The  second  part  of  the  right  hand  side 
in  Eq.  (1 ),  e  =  [e(x1),e(x2),...,e(x„)]r ,  is  a  realization  of  the  stochastic  process  e(\)  that  is  assumed  to  have  zero 
mean  and  covariance  structure  E[e(x,)e(Xy)]  =  <r2i?(0,x;,x.)  ,  where  cr~  is  the  process  variance  and  0  is  the 

process  correlation  parameter.  The  optimal  choice  of  0  is  defined  as  the  maximum  likelihood  estimator  (MLE), 
which  is  the  maximizer  of  the  likelihood  function  Z,,  expressed  as 

_n  l 

L  =(2;rcr2)  2  |R|  2  exp 

where  R  is  the  symmetric  correlation  matrix  with  i-jlh  component  R  =R(Q,xj,xj),  i,j  =  1, ...,«,  and 

tr2  =  — (Y  — Fp)TR  '(Y  — Fp)  and  p  =  (ftR  ‘f)  ‘  FTR  Y  are  obtained  from  the  generalized  least  square 
n  v  ' 

regression. 

Under  the  general  decomposition  of  Eq.  (1),  the  objective  is  to  predict  the  noise- free  unbiased  response  at  a  new 
point  of  interest  x,  expressed  as 

yt„;,(x)  =  flp  +  rl  R  (Y-Fp)  (3) 

where  r  =  [/?(0,x1,x),...,/?(0,xJ1,x)]T  .  The  prediction  variance  can  be  also  obtained  as 

cr2  (x)  =  CT2  (1  +  uT (F’R-'F)"1  u  -r TR  r) 


--±t(Y-FP)tR  ‘(Y-Fp) 

Id 


(2) 


where  u  =  F2R  ‘r-f. 


(4) 


In  the  dynamic  Kriging  method,  the  basis  function  set  f(x)  is  no  longer  fixed.  Instead,  the  optimal  basis  function 
set  is  decided  by  the  given  samples.  First,  a  number  of  polynomial  functions  are  considered  as  the  candidate 
function.  The  highest  order  P  of  the  polynomial  is  decided  by  (  Fnd+P  )  <  n  - 1 .  A  genetic  algorithm  is  used  to  find 

the  optimal  subset  of  the  basis  function  by  minimizing  the  process  variance  a~ .  Second,  a  generalized  pattern 
search  algorithm  is  used  to  solve  the  problem  that  is  maximizing  Eq.  (2).  With  these  two  steps  carried  out,  the 
dynamic  Kriging  can  generate  a  more  accurate  surrogate  model  than  the  traditional  universal  Kriging  [3]. 


4.2  Sampling-based  RBDO 

The  mathematical  formulation  of  a  general  RBDO  problem  is  expressed  as 

minimize  Cost(d) 

subjectto  P[Gj(X)>0]<PF“r,  j  =  l,---,nc  (5) 

dL  <  d  <  du,  demand  XeR"' 

where  d  =  {dj  }T  =  p(Xrv ),  i  =  1  ~  nd  is  the  design  vector,  which  is  the  mean  value  of  the  nrf-dimensional  random 
variable  vector  Xrv={X,,  Xnd}T  ;  X={Xrv,Xrp}T  where  Xrv  and  Xrp  stand  for  the  random  design 

variable  and  random  parameter  components  of  the  random  input  X ,  respectively;  Pj"'  is  the  target  probability  of 

failure  for  the /h  constraint;  and  nc,  nd,  and  nr  are  the  number  of  probabilistic  constraints,  design  variables,  and 
random  variables  plus  parameters,  respectively. 

A  reliability  analysis  for  both  the  component  and  system  levels  involves  calculation  of  the  probability  of  failure, 
denoted  by  PF,  which  is  defined  using  a  multi-dimensional  integral  as 

PFi'V)  =  P[X  efif]  =  JR„  Iap  (x)/x (x;  \\i)dx  =  E  [/Q(.  (X)]  (6) 

where  v|/  is  a  vector  of  distribution  parameters,  which  usually  includes  the  mean  (p)  and/or  standard  deviation  (a) 


of  the  random  input  X  =  {jr1,—,Zjr}T;  P[»]  represents  a  probability  measure;  fi,  is  the  failure  set;  /X(x;v|/)  is 
a  joint  probability  density  function  (PDF)  of  X;  and  £[•]  represents  the  expectation  operator.  The  failure  set  is 
defined  as  Q.F  =  jx :  G^.(x)  >  dj  for  component  reliability  analysis  of  the  fb  constraint  function  G,{x),  and 


QF  =  •■  x  :  ^ Gj(x)  >  oj  and  QF  =  |x :  P|  *^Gy.(x)  >  o|  for  the  series  system  and  parallel  system  reliability 

analysis  of  nc  perfonnance  functions,  respectively.  Ia  (x)  in  Eq.  (6)  is  called  an  indicator  function  and  defined  as 

fl,  xeQ, 

4  (xH  (7) 

F  [O,  otherwise 

In  this  paper,  since  the  mean  of  X,  p  =  {//,,■  ■•,/rw}T  is  used  as  a  design  vector,  the  vector  of  distribution 
parameters  v|/  is  simply  replaced  with  p  for  the  computation  of  the  probability  of  failure  in  Eq.  (6).  Taking  the 
partial  derivative  of  probability  of  failure  in  Eq.  (6)  with  respect  to  the  1th  design  variable  //,  yields 


dPF(p) 

Bp, 


|r„4/x)./x('x;p)^x 


and  the  differential  and  integral  operators  can  be  interchanged  using  the  Leibniz’s  rule,  giving 


dPF0i) 

Bp, 


=  E 


Lvx) 

4f(x) 


^/x(x;r) 

Bp, 

51n/x(x;p) 


£/x  =  Jr„4/x) 


gln/x(x;p) 

Bp, 


/X(x;p)c/x 


Bp, 


(8) 


(9) 


since  Ia  (x)  is  not  a  function  of  p{ .  The  partial  derivative  of  the  log  function  of  the  joint  PDF  in  Eq.  (8)  with 


respect  to  p,  is  known  as  the  first-order  score  function  for  pt  and  is  denoted  as 


Bp, 


(10) 


To  compute  the  probability  of  failure  in  Eq.  (5)  and  the  sensitivity  of  probability  of  failure  in  Eq.  (8),  statistical 
sampling  such  as  the  Monte  Carlo  simulation  (MCS)  at  a  given  design  needs  to  be  applied  to  true  responses,  which 
is  computationally  very  expensive  and  almost  prohibited.  Hence,  instead  of  using  true  responses,  which  are  usually 


obtained  from  computer  simulations,  surrogate  models  are  used  be  implemented  for  the  calculation  of  the 
probability  of  failure.  To  generate  accurate  surrogate  models,  this  paper  uses  the  dynamic  Kriging  method,  which 
is  discussed  in  the  previous  section. 

Denote  the  surrogate  model  obtained  by  the  dynamic  Kriging  method  for  the  constraint  function  G;(X)  as  G  (X)  . 


Then,  by  carrying  out  the  MCS  using  the  conservative  surrogate  model  G,  (X)  ,  the  probabilistic  constraints  in  Eq. 
(5)  can  be  approximated  as 


where  Mis  the  MCS  sample  size, 
defined  as  ClF  =  jx  :  Gf(x)  >  oj . 


1  M 

PF  -  P[Gj (X)  >  0]  =  —  (X,m) )  <  Pf 

1  M  ,  F • 

-Lrj-  m= 1 


iTar 

F, 


x(m)  is  the  m!h  realization  of  X,  and  the  failure  set  ClF 
Sensitivity  of  the  probabilistic  constraint  in  Eq.  (8)  is 

dPF  1  « 

dft,  * 


(ID 

for  the  surrogate  model  is 
obtained  as 


(12) 


where  .s^l,(x("l);p)  is  obtained  using  Eq.  (9). 


4.3  Corrected  Akaike  information  criterion  (AICc) 

AIC  is  originally  proposed  to  evaluate  the  quality  of  a  model  in  statistics  based  on  the  log-likelihood  function  [10]. 
It  is  a  measure  of  the  relative  goodness  of  fit  of  a  statistical  model,  and  in  general  expressed  as 

AIC  =  2k-2ln(L)  (13) 


where  k  is  the  number  of  parameters  in  the  statistical  model  and  L  is  the  maximized  value  of  the  likelihood  function 
for  the  estimated  model.  In  the  Kriging  framework  discussed  in  Section  4.1,  the  k  is  the  number  of  dimension  of  X, 
which  is  nd,  and  L  is  the  likelihood  function  as  shown  in  Eq.  (2).  When  the  sample  size  is  small,  i.e.,  nlk  <  40  (n  is 
the  number  of  sample),  which  is  often  the  case  in  engineering  problems,  the  corrected  AIC  (AICc)  is  used  instead 
to  provide  an  unbiased  estimation,  which  is  expressed  as 


AICc  =  AIC  + 


2k{k  +  \) 
n-k- 1 


(14) 


5  Sampling-based  RBDO  Using  Conservative  Surrogate  Model 

5.1  Weighted  Kriging  Prediction  Variance  for  Conservative  Surrogate  Model 

In  Eq.  (4),  the  Kriging  prediction  variance  <J2p  is  calculated,  and  the  C%  upper  bound  of  the  prediction  interval  is 


y 


var 


:  Tkri: 


+  4>~ 


(15) 


where  <f>~'  (•)  is  the  inverse  CDF  of  the  standard  normal  random  variable.  This  prediction  upper  bound  is  usually 

used  as  the  C%  level  conservative  surrogate  model  for  the  Kriging  method,  and  often  it  is  considered  as  a  variable 
safety  margin  approach  for  constructing  the  conservative  surrogate  model.  Since  the  Kriging  method  is  an 
interpolation  method,  the  prediction  variance  cr2  becomes  zero  at  the  sample  point.  As  a  result,  the  conservative 


surrogate  model  yvar  is  the  same  as  the  ytrig  at  the  sample  points.  This  interpolation  property  of  the  Kriging 

method  causes  trouble  when  the  conservative  surrogate  model  is  used  for  an  optimization  problem.  Consider  a  1-D 
example, 

miny  = -(6x-2)2  sin(12x-4)  xe[0,l] 

The  function  plot  and  the  Kriging  prediction  using  seven  points  are  shown  in  Fig.  1 . 


(16) 
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Figure  1 :  1-D  problem  with  7  samples 

When  the  upper  bound  is  used  as  the  conservative  surrogate  model  and  for  optimization,  the  optimum  x  could  be 
easily  converged  to  the  sample  points,  which  are  the  local  minima  of  yvar .  Therefore,  this  upper  bound  of  the 
Kriging  prediction  interval  is  not  applicable  for  optimization.  Another  constant  safety  margin  approach  is  also 
often  used  for  the  conservative  surrogate  model  [7],  where  the  safety  margin  is  decided  based  on  the  empirical 
CDF  of  the  cross-validation  error.  In  this  constant  safety  margin  approach,  the  conservative  surrogate  model  is 
obtained  by  shifting  the  Kriging  prediction  to  a  certain  amount  and  is  often  expressed  as 

Txv  =  Tkrig  "*"eXF,%C  (17) 

where  exv  %c  is  the  C  percentile  of  the  cross-validation  error.  This  conservative  surrogate  model  has  the  same 

discrepancy  from  the  Kriging  prediction  regardless  of  the  sample  position.  This  raises  the  problem  that  the 
conservative  surrogate  model  cannot  count  in  the  different  uncertainty  of  the  discrepancy  due  to  different  sample 
locations.  Therefore,  this  constant  safety  margin  approach  may  become  over-conservative  if  the  samples  are  dense 
and  under-conservative  if  the  samples  are  sparse.  Consider  the  same  example  shown  in  Fig.  1 .  If  there  is  no  sample 
at  x  =  0.76,  the  safety  margin  Eq.  (17)  is  s  =  1.454  and  the  conservative  surrogate  model  is  shown  in  Fig.  2.  The 
conservative  surrogate  model  is  over-conservative  in  the  region  of  [0,  0.55]  and  under-conservative  in  the  region  of 
[0.55,  1], 


Figure  2:  1-D  problem  with  6  samples 


According  to  the  two  examples  discussed  above,  one  can  see  that  when  a  conservative  surrogate  model  from  the 
Kriging  method  is  used  for  an  optimization  problem,  a  variable  safety  margin  approach  that  has  zero  margin  at  the 
sample  points  is  not  desirable  because  it  generates  unnecessary  local  optima  regions;  whereas  a  constant  safety 
margin  may  not  be  desirable  because  it  does  not  count  in  the  effect  from  the  sample  position.  What  is  needed  is  a 
conservative  surrogate  model  that  uses  a  variable  safety  margin  that  does  not  generate  local  optima  and  counts  in 
the  effect  from  the  sample  position. 

In  this  paper,  a  new  conservative  surrogate  model  that  combines  the  variable  safety  factor  approach  from  Eq.  (15) 
and  the  constant  safety  margin  approach  from  Eq.  (17)  is  proposed.  First,  to  count  in  the  effect  from  the  sample 
position,  an  importance  measure  for  each  sample  is  needed.  In  this  paper,  the  AICc  is  chosen  to  quantify  how 
important  a  sample  is  for  the  Kriging  prediction.  The  importance  function  for  sample  jc;  is  defined  as 


Im(x;) : 


AICc -AICc (-° 


AICc 


,i  =  l,...,« 


(18) 


where  AICc  is  calculated  using  all  n  samples  and  AICc{~:)  is  calculated  by  omitting  x;.  From  Eq.  (18),  it  shows  that 
the  larger  Im(x;)is,  the  more  important  Xj  is.  Consider  the  same  example  discussed  above.  Figs.  3(a)-(g)  show  the 
Kriging  predictions  when  one  of  the  samples  is  omitted,  and  the  associated  importance  function  values  are  shown 
in  Fig.  3(h)  as  well.  The  sample  point  6  at  x  =  0.76  is  indeed  around  the  highly  nonlinear  region,  and  its  importance 
function  value  is  the  largest  among  all  7  samples.  Therefore,  this  importance  function  by  the  relative  change  in 
AICc  values  can  characterize  how  important  one  sample  is  for  the  Kriging  prediction  accuracy. 


(a)  Kriging  prediction  w/o  sample  #2 


(c)  Kriging  prediction  w/o  sample  #4 


(b)  Kriging  prediction  w/o  sample  #3 


(f)  Kriging  prediction  w/o  sample  #5 


•16 


0  82 


(g)  Kriging  prediction  w/o  sample  #6 


(h)  Im(x)  function  values  at  each  sample 


Figure  3:  Leave-one-out  Kriging  prediction 


After  the  importance  for  each  sample  is  calculated,  to  construct  a  weighted  Kriging  variance,  a  leave-one-out 
Kriging  variance  (crY  'is  calculated  where  the  z'th  sample  X;  is  omitted  using  Eq.  (4),  and  then  the  weighted 
Kriging  variance  is  expressed  as 


o' 


p,  weighted 


where  the  weight  function  is  defined  as 


w(x,.)  = 


1  /  Im(x,. ) 
£l/Im(x,) 


(19) 

(20) 


The  weight  function  decides  how  much  the  leave-one -out  Kriging  variance  (cr  Y 

Kriging  variance  when  Xj  is  missing  according  to  the  importance  function  value, 
surrogate  model  based  on  the  weighted  Kriging  variance  is 


v  =  VL,  +d) 

•z  con  y  krig 


C 

100 


o 


p,  weighted 


contributes  to  the  total 
Finally,  the  conservative 

(21) 


According  to  Eq.  ( 1 9),  one  can  see  that  when  x;  is  omitted,  the  leave-one-out  Kriging  variance  (o2p )'  '  would  not 
become  zero  at  xj,  and  therefore  the  total  Kriging  variance  would  not  be  zero  at  Xj  and  indeed  smoothes  the 
conservative  surrogate  model  eventually.  On  the  other  hand,  since  (cr)'  "  will  be  larger  at  the  place  where  no 

sample  is  nearby,  it  will  make  the  total  weighted  Kriging  variance  become  variable  among  the  entire  domain. 
Consider  the  same  example  shown  in  Figs.  4  and  5.  In  Fig.  4,  the  lines  labeled  as  Y_true  ,  Y_krig,  Y_xv,  , 
Y_var  and  Y_con  are  the  true  response,  the  Kriging  response,  the  conservative  surrogate  model  using  constant 
safety  margin,  the  upper  bound  of  Kriging  prediction  interval,  and  the  conservative  surrogate  model  using  the 
weighted  Kriging  variance,  respectively. 

When  all  seven  samples  are  used  and  the  conservativeness  is  set  to  be  90%,  the  conservative  surrogate  model  using 
the  weighted  Kriging  variance  is  smoother  than  the  upper  bound  of  the  Kriging  prediction  interval  and  closer  to  the 
true  response  than  the  conservative  surrogate  model  using  the  safety  margin.  Moreover,  if  one  important  sample 
(i.e.,  x  =  0.76  in  this  case)  is  missing,  the  conservative  surrogate  model  using  the  weighted  Kriging  variance  would 
have  a  larger  discrepancy  than  the  conservative  surrogate  model  using  the  constant  safety  margin  approach  in  the 
region  of  [0.55,  1]  and  a  smaller  discrepancy  in  the  region  of  [0,  0.55],  as  shown  in  Fig.  5.  These  two  cases  indicate 
that  the  conservative  surrogate  model  using  the  weighted  Kriging  variance  can  adaptively  identify  the 
conservativeness  according  to  sample  locations  and  provide  a  smooth  surrogate  model  that  does  not  change  the 
optimum  region  of  the  original  response  function. 
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Figure  4:  Conservative  surrogate  model  using  weighted  Kriging  variance  (7  samples) 


Figure  5:  Conservative  surrogate  model  using  weighted  Kriging  variance  (6  samples) 

5.2.  Sampling-based  RBDO  using  the  conservative  surrogate  model 

When  a  limited  number  of  samples  is  used  to  generate  the  surrogate  model  for  MCS  in  sampling-based  RBDO,  to 
assure  the  optimal  design  can  satisfied  the  probabilistic  constraints,  the  surrogate  models  in  Eqs.  ( 1 1 )  and  ( 1 2)  need 
to  be  replaced  by  the  conservative  surrogate  model.  Therefore,  the  conservative  surrogate  model  using  the 
weighted  Kriging  variance  from  Eq.  (21)  is  used  to  represent  the  original  performance  function.  The  formulation  of 
sampling-based  RBDO  becomes 

minimize  Cost(d) 

subject  to  P[G'j  (X)  >  0]  <  PFTar ,  j  =  \,---,nc  (22) 

dL  <  d  <  du,  d  e  R"*'  and  X  e  R ' 

where  G'j  (X)  is  the  conservative  surrogate  model  for  representing  performance  function  G(X)  . 

6.  Numerical  Example 

6.1  2-D  RBDO  Problem  with  Highly  Nonlinear  Constraint  Functions 
Consider  a  2-D  RBDO  problem  with  three  probabilistic  constraints,  expressed  as 


minimize 
subject  to 


Cost(d)  =  - 


(rf,+rf2-10)2  (dx-d2+\Qf 


30  120 

P(G,.(X(d))  >  0)  <  PH*  =  2.275%,  j  =  1  ~  3 

dL<d<dr,  d  e  R2  andX  e  R2 


(23) 


where  three  constraints  functions  are 


G,(X)  =  1-^A 
20 

G,(X)  =  -l  +  (7-6)2  +  (7-6)3  -0.6x(7-6)4  +Z 
80 


(24) 


G3(X)  =  1- 


V,2+8V,  +5 


where 


0.9063  0.4226 
0.4226  -0.9063 


1  >  and  are  drawn  in  Fig.  6.  The  distribution  and  design  domain  for  each 


variable  are  shown  in  Table  1.  The  initial  design  point  is  d°  =  [5  5], 

Table  1:  Properties  of  Random  Variables 


Random 

Variables 

Distribution 

dL 

d° 

du 

Standard 

Deviation 

X, 

Normal 

0.0 

5.0 

10.0 

0.3 

X2 

Normal 

0.0 

5.0 

10.0 

0.3 

Figure  6:  Cost  and  constraint  functions  plot 

It  is  worth  mentioning  that  there  is  no  need  to  construct  the  conservative  surrogate  model  from  the  initial  design 
point.  In  the  proposed  RBDO  process,  the  deterministic  design  optimization  is  carried  out  first.  The  RBDO  process 
starts  from  the  detenninistic  optimum  thereafter.  The  conservative  surrogate  model  will  be  constructed  for  the 
RBDO  process  only.  In  this  example,  the  deterministic  optimum  is  x  =  [5.19,  0.74]  (which  is  the  magenta  cross  in 
Fig.  7),  and  the  number  of  samples  in  the  local  window  is  fixed  to  10  samples  from  the  Latin  hypercube  sampling 
method.  Since  10  samples  may  not  be  enough  to  construct  the  accurate  surrogate  models,  the  conservative 
surrogate  models  using  the  proposed  weighted  Kriging  variance  from  Eq.  (21)  are  generated  for  two  active 
constraints  and  G2.  For  comparison  study,  the  conservative  surrogate  models  using  the  constant  safety  margin 
from  Eq.  (17)  are  also  generated.  The  sampling-based  RBDO  is  carried  out  using  these  two  different  conservative 
surrogate  models,  and  the  optimum  designs  are  compared  in  Table  2  and  plotted  in  Fig.  7.  The  C%  level  is  set  to  be 
90%  in  this  example. 

In  Fig.  7,  it  can  be  seen  that  the  surrogate  model  from  the  dynamic  Kriging  model  itself,  which  is  the  blue  line, 


underestimates  the  true  response  and  results  in  danger  for  the  obtained  optimum  design  d  =  [4.7432,  1.5391]  (blue 
cross  in  Fig.  7)  as  the  probability  of  failure  for  G2  is  2.4992%,  which  is  larger  than  the  target  probability  of  failure 
2.275%,  as  shown  in  Table  2.  Therefore  the  conservative  surrogate  model  is  indeed  necessary  for  counting  in  the 
uncertainty  from  the  surrogate  model  and  assuring  that  the  optimum  design  can  satisfy  the  probabilistic  constraints. 
By  using  the  weighted  Kriging  variance  for  the  conservative  surrogate  model,  which  is  the  red  line  in  Fig. 7,  the 
obtained  RBDO  optimum  design  is  d  =  [4.7020,  1.5801],  which  is  the  red  cross  in  Fig.  7,  and  the  cost  function 
value,  the  probability  of  failure  for  Gh  and  the  probability  of  failure  for  G2  are  -1.8956,  2.0654%,  and  1.6313%, 
respectively.  As  a  comparison,  the  constant  safety  margin  approach  gives  a  more  conservative  optimum  design  d  = 
[4.6510,  1.6205]  (the  green  cross  in  Fig.  7)  where  the  probability  of  failure  for  G\  and  G2  are  1.8354%  and 
0.9455%,  respectively;  and  the  cost  function  value  is  -1.8783.  From  the  results  in  Table  2,  one  can  see  that  the 
conservative  surrogate  model  using  the  weighted  Kriging  variance  can  assure  the  optimum  design  satisfying  the 
probabilistic  constraints  and  has  a  better  optimum  design  in  terms  of  cost  function  compared  to  the  conservative 
surrogate  model  using  the  constant  safety  margin  approach. 


Figure  7:  Different  conservative  surrogate  models  and  optimum  designs 
Table  2:  Optimum  designs  from  different  surrogate  models  and  the  probability  of  failure  at  the  optimum 


Methods 

Cost 

Optimum  Design 

MCS  (5M) 

Ph  - ,  % 

pF2,% 

Surrogate 

Model 

Dynamic  Kriging 

-1.9136 

4.7432,  1.5391 

2.3256 

2.4992 

Constant  Safety  Margin 

-1.8783 

4.6510,  1.6205 

1.8354 

0.9455 

Weighted  Kriging  Variance 

-1.8956 

4.7020,  1.5801 

2.0654 

1.6313 

Analytical 

-1.9079 

4.7351,  1.5518 

2.2715 

2.2633 

When  using  the  surrogate  model  for  sampling-based  RBDO,  the  sample  profile  may  affect  the  result  as  well.  A 
good  sample  profile  for  dynamic  Kriging  may  end  up  having  a  better  surrogate  model.  Therefore,  to  investigate 
whether  the  proposed  weighted  Kriging  variance  for  the  conservative  surrogate  model  has  a  stable  performance,  a 
statistical  study  is  carried  out.  In  this  statistical  study,  50  sets  of  10-LHS  samples  are  generated.  For  each  sample 
set,  the  sampling-based  RBDO  is  carried  out  using  both  the  weighted  Kriging  variance  approach  and  the  constant 
safety  margin  approach.  The  comparison  for  cost  function  value  and  the  probability  of  failure  at  the  optimum  are 
shown  in  Table.  3. 

Table:  3  Cost  function  and  probability  of  failure  at  optimum  design  (50  trials) 


Methods 

Cost 

P*  ,  % 

PFl ,  % 

#  of  Violation 

#  of  Violation 

(Median) 

(Median) 

(Median) 

G\ 

g2 

Dynamic  Kriging 

-1.9164 

2.2982 

2.3232 

1 

28 

Constant  Safety  Margin 

-1.8809 

1.9002 

0.9519 

0 

1 

Weighted  Kriging  Variance  -1.8981 


2.1352 


1.6287 


0 


3 


In  Table  3,  when  using  the  dynamic  Kriging  method  itself,  the  optimum  designs  violate  the  probabilistic  constraint 
G2  28  times  out  of  50  trials.  It  shows  that  the  dynamic  Kriging  prediction  either  underestimates  the  true  response  or 
overestimates  the  true  response  at  a  rough  50%  chance  for  each  way.  By  using  the  weighted  Kriging  variance  for 
the  conservative  surrogate  model,  the  obtained  optimum  designs  violate  the  probabilistic  constraint  G2  three  times. 
As  a  comparison,  if  the  constant  safety  margin  is  used  for  the  conservative  surrogate  model,  the  number  of 
violations  for  G2  is  1  time  out  of  50  trials.  Constraint  Gt  is  not  violated  in  both  conservative  surrogate  modeling 
cases  due  to  its  linearity.  The  median  cost  function  value  at  the  optimum  (which  is  -1.8809)  using  the  constant 
safety  margin  is  larger  than  the  one  by  using  the  weighted  Kriging  variance,  which  is  (-1.8981).  It  is  worth 
mentioning  that  since  the  conservativeness  level  in  this  example  is  set  to  be  90%,  both  the  constant  safety  margin 
approach  and  the  weighted  Kriging  variance  approach  satisfy  the  conservativeness.  However,  the  weighted 
Kriging  variance  approach  provides  a  better  optimum  in  terms  of  the  cost  function  value. 

7.  Conclusion 

When  applying  the  surrogate  model  for  optimization  problems,  the  conservative  surrogate  model  using  the  upper 
bound  of  the  Kriging  prediction  interval  is  not  desirable  because  it  generates  multiple  local  optimums  at  the  sample 
points.  The  conservative  surrogate  model  using  the  constant  safety  margin  does  not  distinguish  the  uncertainty  of 
the  surrogate  model  at  different  sample  locations,  and  often  the  obtained  optimum  becomes  over-conservative 
where  samples  are  dense  and  under-conservative  where  samples  are  sparse.  A  weighted  Kriging  variance  using  the 
changes  in  AICc  of  the  Kriging  model  to  quantify  the  importance  from  each  sample  point  is  proposed  to  construct 
a  conservative  surrogate  model  that  does  not  generate  unnecessary  local  optimum  and  distinguishes  the  uncertainty 
of  the  surrogate  model  at  different  sample  locations.  By  applying  the  weighted  Kriging  variance  for  the 
conservative  surrogate  model  in  the  sampling-based  RBDO  problem,  the  obtained  optimum  satisfies  the 
probabilistic  constraints  at  the  conservativeness  level  and  achieves  a  better  optimum  in  terms  of  cost  function  value 
compared  with  the  optimum  using  the  constant  safety  margin. 
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